Search Results: "blade"

22 December 2014

Jonathan Dowland: Blade Runner: Alien Easter Egg?

A few weeks ago I went to see Blade Runner: The Final Cut in a one-off showing at the Tyneside Cinema. I've only watched the Final Cut once and that viewing was a bit compromised, so it was nice to see it properly, and on a massive screen with decent surround sound too. Whilst watching it I saw something that I thought might potentially have been a visual reference to Scott's earlier movie, Alien. When Deckard is climbing onto the roof of the Bradbury building, there's a decorative motif that to me, looks very Giger-esque, biomechanical, a bit like a chest-burster.
click for full frame click for full frame
I went back and took a screenshot from the Blu Ray at the same point. What do you think? As it happens, whilst the inner regions of the building in the movie are the Bradbury building (or at least the entrance hall), I don't think the upper exterior shots are.
Purge Purge
There are a few other, well-known references to Alien in Blade Runner, although they are likely simply re-used effects rather than explicitly easter eggs. The display screen in Gaff's Spinner is re-used from the Narcissus (here's a comparison) and the ambience in Deckard's apartment also featured in the Nostromo's medical bay. Some enterprising person has put together a 12 hour session of that particular sound effect looping. There's a number of other comparisons and spots at this propsummit.com thread.

16 November 2014

John Goerzen: Contemplative Weather

Sometimes I look out the window and can t help but feel this weather is deep. Deep with meaning, with import. Almost as if the weather is confident of itself, and is challenging me to find some meaning within it. This weekend brought the first blast of winter to the plains of Kansas. Saturday was chilly and windy, and overnight a little snow fell. Just enough to cover up the ground and let the tops of the blades of grass poke through. Just enough to make the landscape look totally different, without completely hiding what lies beneath. Laura and I stood silently at the window for a few minutes this morning, gazing out over the untouched snow, extending out as far as we can see. Yesterday, I spent some time with my great uncle and aunt. My great uncle isn t doing so well. He s been battling cancer and other health issues for some time, and can t get out of the house very well. We talked for an hour and a half about news of the family, struggles in life now and in the past, and joys. There were times when all three of us had tears in our eyes, and times when all of us were laughing so loudly. My great uncle managed to stand up twice while I was there this took quite some effort once to give me a huge hug when I arrived, and another to give me an even bigger hug when I left. He has always been a person to give the most loving hugs. He hadn t been able to taste food for awhile, due to treatment for cancer. When I realized he could taste again, I asked, When should I bring you some borscht? He looked surprised, then got a huge grin, glanced at his watch, and said, Can you be back by 3:00? His brother, my grandpa, was known for his beef borscht. I also found out my great uncle s favorite kind of bread, and thought that maybe I would do some cooking for him sometime soon. Today on my way home from church, I did some shopping. I picked up the ingredients for borscht and for bread. I came home, said hi to the cats that showed up to greet me, and went inside. I turned on the radio Prairie Home Companion was on and started cooking. It takes a long time to prepare what I was working on I spent a solid two hours in the kitchen. As I was chopping up a head of cabbage, I remembered coming to what is now my house as a child, when my grandpa lived here. I remembered his borscht, zwiebach, monster cookies; his dusty but warm wood stove; his closet with toys in it. I remembered two years ago, having nearly 20 Goerzens here for Christmas, hosted by the boys and me, and the 3 gallons of borscht I made for the occasion. I poured in some tomato sauce, added some water. The radio was talking about being kind to people, remembering that others don t always have the advantages we do. Garrison Keillor s fictional boy in a small town, when asked what advantages he had, mentioned belonging. Yes, that is an advantage. We all deal with death, our own and that of loved ones, but I am so blessed by belonging to a loving family, two loving churches, a wonderful community. Out came three pounds of stew beef. Chop, chop, slice, plunk into the cast iron Dutch oven. It s my borscht pot. It looks as if it would be more at home over a campfire than a stovetop, but it works anywhere. Outside, the sun came up. The snow melts a little, and the cats start running around even though it s still below freezing. They look like they re having fun playing. I m chopping up parsley and an onion, then wrapping them up in a cheesecloth to make the spice ball for the borscht. I add the basil and dill, some salt, and plonk them in, too. My 6-quart pot is nearly overflowing as I carefully stir the hearty stew. On the radio, a woman who plays piano in a hospital and had dreamed of being on that particular radio program for 13 years finally was. She played with passion and delight I could hear through the radio. Then it s time to make bread. I pour in some warm water, add some brown sugar, and my thoughts turn to Home On The Range. I am reminded of this verse:
How often at night when the heavens are bright
With the light from the glittering stars
Have I stood here amazed and asked as I gazed
If their glory exceeds that of ours.
There s something about a beautiful landscape out the window to remind a person of all the blessings in life. This has been a quite busy weekend actually, a busy month but despite the fact I have a relative that is sick in the midst of it all, I am so blessed in so many ways. I finish off the bread, adding some yeast, and I remember my great uncle thanking me so much for visiting him yesterday. He commented that a lot of younger people have no use for visiting an old geezer like me. I told him, I ve never been like that. I am so glad I could come and visit you today. The best gifts are those that give in both directions, and this surely is that. Then I clean up the kitchen. I wipe down the counters from all the bits of cabbage that went flying. I put away all the herbs and spices I used, and finally go to sit down and reflect. From the kitchen, the smells of borscht and bread start to seep out, sweeping up the rest of the house. It takes at least 4 hours for the borscht to cook, and several hours for the bread, so this will be an afternoon of waiting with delicious smells. Soon my family will be home from all their activities of the day, and I will be able to greet them with a warm house and the same smells I stepped into when I was a boy. I remember this other verse from Home On the Range:
Where the air is so pure, the zephyrs so free,
The breezes so balmy and light,
That I would not exchange my home on the range
For all of the cities so bright.
Today s breeze is an icy blast from the north maybe not balmy in the conventional sense. But it is the breeze of home, the breeze of belonging. Even today, as I gaze out at the frozen landscape, I realize how balmy it really is, for I know I wouldn t exchange my life on the range for anything.

23 October 2014

Alessio Treglia: Bits from the Debian Multimedia Maintainers

This brief announcement was released yesterday to the debian-devel-announce mailing list. Ciao! The Debian Multimedia Maintainers have been quite active since the Wheezy release, and have some interesting news to share for the Jessie release. Here we give you a brief update on what work has been done and work that is still ongoing. Let s see what s cooking for Jessie then. Frameworks and libraries Support for many new media formats and codecs. The codec library libavcodec, which is used by popular media playback applications including vlc, mpv, totem (using gstreamer1.0-libav), xine, and many more, has been updated to the latest upstream release version 11 provided by Libav. This provides Debian users with HEVC playback, a native Opus decoder, Matroska 3D support, Apple ProRes, and much more. Please see libav s changelog for a full list of functionality additions and updates. libebur128 libebur128 is a free implementation of the European Broadcasting Union Loudness Recommendation (EBU R128), which is essentially an alternative to ReplayGain. The library can be used to analyze audio perceived loudness and subsequentially normalize the volume during playback. libltc libltc provides functionalities to encode and decode Linear (or Longitudinal) Timecode (LTC) from/to SMPTE data timecode. libva libva and the driver for Intel GPUs has been updated to the 1.4.0 release. Support for new GPUs has been added. libva now also supports Wayland. Pure Data A number of new additional libraries (externals) will appear in Jessie, including (among others) Eric Lyon s fftease and lyonpotpourrie, Thomas Musil s iemlib, the pdstring library for string manipulation and pd-lua that allows to write Pd-objects in the popular lua scripting language. JACK and LADI LASH Audio Session Handler was abandoned upstream a long time ago in favor of the new session management system, called ladish (LADI Session Handler). ladish allows users to run many JACK applications at once and save/restore their configuration with few mouse clicks. The current status of the integration between the session handler and JACK may be summarized as follows: Note that ladish uses the D-Bus interface to the jack daemon, therefore only Jessie s jackd2 provides support for and also cooperates fine with it. Plugins: LV2 and LADSPA Debian Jessie will bring the newest 1.10.0 version of the LV2 technology. Most changes affect the packaging of new plugins and extensions, a brief list of packaging guidelines is now available.
A number of new plugins and development tools too have been made available during the Jessie development cycle: LV2 Toolkit LVTK provides libraries that wrap the LV2 C API and extensions into easy to use C++ classes. The original work for this was mostly done by Lars Luthman in lv2-c++-tools. Vee One Suite The whole suite by Rui Nuno Capela is now available in Jessie, and consists of three components: All three are provided in both forms of LV2 plugins and stand-alone JACK client. JACK session, JACK MIDI, and ALSA MIDI are supported too. x42-plugins and zam-plugins LV2 bundles containing many audio plugins for high quality processing. Fomp Fomp is an LV2 port of the MCP, VCO, FIL, and WAH plugins by Fons Adriaensen. Some other components have been upgraded to more recent upstream versions: We ve packaged ste-plugins, Fons Adriaensen s new stereo LADSPA plugins bundle. A major upgrade of frei0r, namely the standard collection for the minimalistic plugin API for video effects, will be available in Jessie. New multimedia applications Advene Advene (Annotate Digital Video, Exchange on the NEt) is a flexible video
annotation application. Ardour3 The new generation of the popular digital audio workstation will make its very first appearance in Debian Jessie. Cantata Qt4 front-end for the MPD daemon. Csound Csound for jessie will feature the new major series 6, with the improved IDE CsoundQT. This new csound supports improved array data type handling, multi-core rendering and debugging features. din DIN Is Noise is a musical instrument and audio synthesizer that supports JACK audio output, MIDI, OSC, and IRC bot as input sources. It could be extended and customized with Tcl scripts too. dvd-slideshow dvd-slideshow consists of a suite of command line tools which come in handy to make slideshows from collections of pictures. Documentation is provided and available in /usr/share/doc/dvd-slideshow/ . dvdwizard DVDwizard can fully automate the creation of DVD-Video filesystem. It supports graphical menus, chapters, multiple titlesets and multi-language streams. It supports both PAL and NTSC video modes too. flowblade Flowblade is a video editor like the popular KDenlive based on the MLT engine, but more lightweight and with some difference in editing concepts. forked-daapd Forked-daapd switched to a new, active upstream again dropping Grand Central Dispatch in favor of libevent. The switch fixed several bugs and made forked-daapd available on all release architectures instead of shipping only on amd64 and i386. Now nothing prevents you from setting up a music streaming (DAAP/DACP) server on your favorite home server no matter if it is based on mips, arm or x86! harvid HTTP Ardour Video Daemon decodes still images from movie files and serves them via HTTP. It provides frame-accurate decoding and is main use-case is to act as backend and second level cache for rendering the
videotimeline in Ardour. Groove Basin Groove Basin is a music player server with a web-based user interface inspired by Amarok 1.4. It runs on a server optionally connected to speakers. Guests can control the music player by connecting with a laptop, tablet, or smart phone. Further, users can stream their music libraries remotely.
It comes with a fast, responsive web interface that supports keyboard shortcuts and drag drop. It also provides the ability to upload songs, download songs, and import songs by URL, including YouTube URLs. Groove Basin supports Dynamic Mode which automatically queues random songs, favoring songs that have not been queued recently.
It automatically performs ReplayGain scanning on every song using the EBU R128 loudness standard, and automatically switches between track and album mode. Groove Basin supports the MPD protocol, which means it is compatible with MPD clients. There is also a more powerful Groove Basin protocol which you can use if the MPD protocol does not meet your needs. HandBrake HandBrake, a versatile video transcoder, is now available for Jessie. It could convert video from nearly any format to a wide range of commonly supported codecs. jack-midi-clock New jackd midiclock utility made by Robin Gareus. laborejo Laborejo, Esperanto for Workshop , is used to craft music through notation. It is a LilyPond GUI frontend, a MIDI creator and a tool collection to inspire and help music composers. mpv mpv is a movie player based on MPlayer and mplayer2. It supports a wide variety of video file formats, audio and video codecs, and subtitle types. The project focuses mainly on modern systems and encourages developer activity. As such, large portions of outdated code originating from MPlayer have been removed, and many new features and improvements have been added. Note that, although there are still some similarities to its predecessors, mpv should be considered a completely different program (e.g. lacking compatibility with both mplayer and mplayer2 in terms of command-line arguments and configuration). smtube SMTube is a stand-alone graphical video browser and player, which makes YouTube s videos browsing, playing, and download such a piece of cake.
It has so many features that, we are sure, will make YouTube lovers very, very happy. sonic-visualiser Sonic Visualiser Application for viewing and analysing the contents of music audio files. SoundScapeRenderer SoundScapeRenderer (aka SSR) is a (rather) easy to use render engine for spatial audio, that provides a number of different rendering algorithms, ranging from binaural (headphone) playback via wave field synthesis to higher-order ambisonics. Videotrans videotrans is a set of scripts that allow its user to reformat existing movies into the VOB format that is used on DVDs. XBMC XBMC has been partially rebranded as XBMC from Debian to make it clear that it is changed to conform to Debian s Policy. The latest stable release, 13.2 Gotham will be part of Jessie making Debian a good choice for HTPC-s. zita-bls1 Binaural stereo signals converter made by Fons Adriaensen zita-mu1 Stereo monitoring organiser for jackd made by Fons Adriaensen zita-njbridge Jack clients to transmit multichannel audio over a local IP network made by Fons Adriaensen radium-compressor Radium Compressor is the system compressor of the Radium suite. It is provided in the form of stand-alone JACK application. Multimedia Tasks With Jessie we are shipping a set of multimedia related tasks.
They include package lists for doing several multimedia related tasks. If you are interested in defining new tasks, or tweaking the current, existing ones, we are very much interested in hearing from you. Upgraded applications and libraries What s not going to be in Jessie With the aim to improve the overall quality of the multimedia software available in Debian, we have dropped a number of packages which were abandoned upstream: We ve also dropped mplayer, presently nobody seems interested in maintaining it.
The suggested replacements for users are mplayer2 or mpv. Whilst the former is mostly compatible with mplayer in terms of command-line arguments and configuration (and adds a few new features too), the latter adds a lot of new features and improvements, and it is actively maintained upstream. Please note that although the mencoder package is no longer available anymore, avconv and mpv do provide encoding functionality. For more information see avconv s manual page and documentation, and mpv s encoding documentation. Broken functionalities rtkit under systemd is broken at the moment. Activity statistics More information about team s activity are available. Where to reach us The Debian Multimedia Maintainers can be reached at pkg-multimedia-maintainers AT lists.alioth.debian.org for packaging related topics, or at debian-multimedia AT lists.debian.org for user and more general discussion.
We would like to invite everyone interested in multimedia to join us there. Some of the team members are also in the #debian-multimedia channel on OFTC. Cheers! Alessio Treglia
on behalf of the Debian Multimedia Maintainers

30 June 2014

Jon Dowland: Blade Runner remastered OST

record photo translucent red pressing
Vangelis' soundtrack for the movie Blade Runner is one of my favourite film soundtracks. Its troubled history is reasonably well documented on Wikipedia . At the time of writing, that article doesn't mention the recent 2013 reissue which was remastered and released on Vinyl, and possibly SACD too. Last year I spotted this record and bought it for a friend's birthday. In the year since I've eventually gave into temptation and bought another copy for myself. This was a little unfortunate in terms of timing as my record player has been packed away for two months pending a house move. I did manage to test it out on a Numark PT-01 but it really needs a proper setup to do it justice. In the mean time, one can watch a video of someone else's copy playing: In related news, I've recently enjoyed the BBC's adaptation of Do Androids Dream of Electric Sheep?, as part of their "Dangerous Visions" series. Much like the film, the play elides a lot of material from the book; including Deckard's wife and (more sadly) the entire Buster Friendly / Mercer side-story. However it retains the second police station, which (to me) was the key, classic Dick "What-the..." moment in the novel.

10 February 2014

Mario Lang: Neurofunkcasts

I have always loved Drum and Bass. In 2013 I rediscovered my love for Darkstep and Neurofunk, and found that these genres have developed quite a lot in the recent years. Some labels like Black Sun Empire and Evol Intent produce mixes/sets on a regular basis as podcasts these days. This article aggregates some neurofunk podcasts I like a lot, most recent first. Enjoy 33 hours and 57 minutes of fun with dark and energizing beats. Thanks to BSE Contrax and Evol Intent for providing such high quality sets. You can also see the Python source for the program that was used to generate this page.

11 January 2014

Debian Sysadmin Team: Peter Palfrader: The Debian DNS universe

Abstract I recently moved our primary nameserver from orff.debian.org, which is an aging blade in Greece, to a VM on one of our ganeti clusters. In the process, I rediscovered a lot about our DNS infrastructure. In this post, I will describe the many sources of information and how they all come together.

Introduction The Domain Name System is the hierarchical database and query protocol that is in use on the Internet today to map hostnames to IP addresses, to map the reverse thereof, to lookup relevant servers for certain services such as mail, and a gazillion other things. Management and authority in the DNS is split into different zones, subtrees of the global tree of domain names. Debian currently has a bit over a score of zones. The two most prominents clearly are debian.org and debian.net. The rest is made up of debian domains in various other top level domains and reverse zones, which are utilized in IP address to hostname mappings.

Types and sources of information The data we put into DNS comes from a wide range of different systems:
  • Classical zonefiles maintained in git. This represents the core of our domain data. It maps services like blends.debian.org to static.debian.org or specifies the servers responsible for accepting mail to @debian.org addresses. It also is where all the ftp.CC.debian.org entries are kept and maintained together with the mirror team.
  • Information about debian.org hosts, such as master, is maintained in Debian's userdir LDAP, queryable using LDAP1.
    • This includes first and foremost the host's IP addresses (v4 and v6).
    • Additionally, we store the server responsible for receiving a host's mail in LDAP (mXRecord LDAP attribute, DNS MX record type).
    • LDAP also has some specs on computers, which we put into each host's HINFO record, mainly because we can and we are old-school.
    • Last but not least, LDAP also has each host's public ssh key, which we extract into SSHFP records for DNS.
  • LDAP also has per-user information. Users of debian infrastructure can attach limited DNS elements as dnsZoneEntry attributes to their user2.
  • The auto-dns system (more on that below).
  • Our puppet also is a source of DNS information. Currently it generates only the TLSA records that enable clients to securely authenticate certificates used for mail and HTTPS, similar to how SSHFP works for authenticating ssh host keys.

Debian's auto-dns and geo setup We try to provide the best service we can. As such, our goal is that, for instance, user access to www or bugs should always work. These services are, thus, provided by more than one machine on the Internet. However, HTTP did not specify a requirement for clients to re-try a different server if one of those in a set is unavailable. This means for us that when a host goes down, it needs to be removed from the corresponding DNS entry. Ideally, the world wouldn't have to wait for one of us to notice and react before they can have their service in a working manner. Our solution for this is our auto-dns setup. We maintain a list of hosts that are providing a service. We monitor them closely. Whenever a server goes away or comes back we automatically rebuild the zone that contains the element. This setup also lets us reboot servers cleanly since one of the things we monitor is "is there a shutdown running", we can, simply by issuing a shutdown -r 30 kernel-update, de-rotate the machine in question from DNS. Once the host is back it'll automatically get re-added to the round-robin zone entry. The auto-dns system produces two kinds of output:
  • In service-mode it generates a file with just the address records for a specific service. This snippet is then included in its zone using a standard bind $INCLUDE directive. Services that work like this include bugs and static (service definition for static).
  • In zone-mode, auto-dns produces zonefiles. For each service it produces a set of zonefiles, one for each out of a set of different geographic regions. These individual zonefiles are then transferred using rsync to our GEO-IP enabled nameservers. This enables us to give users a list of security mirrors closer to them and thus hopefully faster for them.

Tying it all together The Debian DNS Rube Goldberg Machine. Figure 1: The Debian DNS Rube Goldberg Machine. Once all the individual pieces of source information have been collected, the dns-update and write_zonefile scripts from our dns-helpers repository take over the job of building complete zonefiles and a bind configuration snippet. Bind then loads the zones and notifies its secondaries. For geozones, the zonefiles are already produced by auto-dns' build-zones and those are pulled from the geo nameservers via rsync over ssh, after an ssh trigger.

and also DNSSEC All of our zones are signed using DNSSEC. We have a script in dns-helpers that produces, for all zones, a set of rolling signing keys. For the normal zones, bind 9.9 takes care of signing them in-process before serving the zones to its secondaries. For our geo-zones we sign them in the classical dnssec-signzone way before shipping them. The secure delegation status (DS set in parent matches DNSKEY in child) is monitored by a set of nagios tests, from both [dsa-nagios] and dns-helpers. Of these, manage-dnssec-keys has a dual job: not only will it warn us if an expiring key is still in the DSset, it can also prevent it from getting expired by issuing timly updates of the keys metadata.

Relevant Git repositories -- Peter Palfrader
  1. ldapsearch -h db.debian.org -x -ZZ -b dc=debian,dc=org -LLL 'host=master'
  2. ldapsearch -h db.debian.org -x -ZZ -b dc=debian,dc=org -LLL 'dnsZoneEntry=*' dnsZoneEntry

9 January 2014

Russ Allbery: Review: Honeyed Words

Review: Honeyed Words, by J.A. Pitts
Series: Sarah Beauhall #2
Publisher: Tor
Copyright: July 2011
ISBN: 0-7653-2907-7
Format: Trade paperback
Pages: 412
Honeyed Words is a direct sequel to Black Blade Blues and probably doesn't make sense to read out of order. Sarah, Katie, their friends, and Katie's family are still recovering from the climactic events of Black Blade Blues and licking their wounds, but things are slowly looking up. Sarah is getting more comfortable openly expressing her sexuality, although there are still tensions, and her relationships are improving. The city of Vancouver also seems to be recovering. But major political shifts among the nonhuman races have echoes and ramifications, and after a kidnapped singer and a murdered dwarf, Sarah is pulled back into the turmoil. If you liked the world-building in Black Blade Blues, Honeyed Words is more of the same, but with more variety. Pitts adds elves to dwarves and dragons, shows considerably more of the supernatural world beneath the surface of human cities, and introduces Sarah to some new types of danger. There are a few more glimmers of the history of Sarah's sword and the nature of the gods in this universe, but still nothing conclusive. Pitts broadens the world-building rather than deepening it. But I liked the new characters. Teenage elves with obsessions make for great characters, and the demon who shows up about halfway through is delightful. As with the first book, this is mostly a story of things happening to Sarah while she tries to muddle through as best she can. There isn't much in the way of grand strategy or of seizing control of the plot. She just tries to do the right thing and gets lots of help from lots of friends. But the plot moves right along, and it's mixed with a healthy sense of normality that I find refreshing. She needs work as a blacksmith, she's trying to support a friend without having much money of her own, the living arrangements this forces are a bit of a strain for her own relationships, and her broken relationship with her family continues to lurk. Much urban fantasy is built around the premise of a regular person thrust into a fantasy world, but much of it then discards normal life once the supernatural starts. I think Pitts does a particularly good job of keeping his protagonist's life grounded in the ongoing mundane problems of living. It's also more fun to read about Sarah when she isn't quite as angstful. Over the course of Honeyed Words, she seems to be growing into a person who can accept who she is and feel secure about her place in the world. That's nice to see. The downside to this book is that, while it's full of quite a lot of stuff, not all of it coheres that horribly well. It has a certain "middle book" feel. The major plot resolves, but with some significant lingering consequences and something of a cliff-hanger last chapter. Most of the threads left open by the first book are still open here. There's character growth, and setup for some resolution, but not a lot of actual resolution. I still like this series, though. I like urban fantasy that isn't about vampires and werewolves; berserker blacksmiths and Norse legend has much more appeal, and I want to read more of it. Pitts's writing is a bit clunky and probably won't win many rewards, but it's serviceable and I think improving. If you enjoyed the first book, it's worth looking for this one as well. Followed by Forged in Fire. Rating: 7 out of 10

2 December 2013

Ben Hutchings: Upgrading from Android 2.3 'Gingerbread' to 4.3 'Jelly Bean'

Replacing my phone My first Android phone was a ZTE Blade (sold as Orange San Francisco here in the UK). It originally shipped with Android 2.1 but was upgradable to 2.3 thanks to CyanogenMod (or other unofficial mods). However there's little sign of an upgrade to 4.x; the apps I want to use are pushing the limits of its CPU, RAM and internal storage; and I've never got very good at typing on a soft keyboard. So it seemed like time to get a newer phone with a hard keyboard (and still with a SD slot). After a little research with the CyanogenMod compatibility list and some time looking at reviews, I settled on the Samsung Galaxy S Relay 4G, which was exclusive to T-Mobile USA but is being resold through eBay. It shipped with Android 4.0 but I immediately installed CyanogenMod 10.2 (Android 4.3). This was a bit of an adventure as I bought the Blade with CM already installed and wasn't familiar with the multiple steps that were necessary. Copying my data The last step, and the real subject of this entry, was to move my data across. Much of this was on a SD card which I could simply plug into the Relay. The internal files had to be backed up onto this card and then restored, using the recovery environment (ClockworkMod) on each phone. After rebooting the Relay into the full Android system, my apps and settings were mostly present but I was immediately confronted with a series of error dialogs reporting that 'Unfortunately, Dialler has stopped' - and the same for 'Clock' and 'the process android.process.acore' (whatever that is). What went wrong There are thankfully more detailed error logs in the filesystem, under /data/system/dropbox (this is for logging at the Java level; if a process is terminated by a fatal signal it's logged to /data/tombstones). There are several ways to get at them, but I used recovery mode and adb to get a root shell where I could easily read and write files as necessary. The Contacts (aka People) database upgrade fails with an SQLiteException, apparently because it doesn't account for upgrading from the schema used in 2.3: 
Process: android.process.acore
Flags: 0x883e45
Package: com.android.providers.applications v18 (4.3.1-f963020e38)
Package: com.android.providers.contacts v18 (4.3.1-f963020e38)
Package: com.android.providers.userdictionary v18 (4.3.1-f963020e38)
Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys
android.database.sqlite.SQLiteException: no such column: phonebook_label (code 1): , while compiling: UPDATE raw_contacts SET display_name_source=?,display_name=?,display_name_alt=?,phonetic_name=?,phonetic_name_style=?,sort_key=?,phonebook_label=?,phonebook_bucket=?,sort_key_alt=?,phonebook_label_alt=?,phonebook_bucket_alt=? WHERE _id=?
...
        at com.android.providers.contacts.ContactsDatabaseHelper.updateRawContactDisplayName(ContactsDatabaseHelper.java:5344)
        at com.android.providers.contacts.ContactsDatabaseHelper.upgradeToVersion504(ContactsDatabaseHelper.java:3580)
        at com.android.providers.contacts.ContactsDatabaseHelper.onUpgrade(ContactsDatabaseHelper.java:2261)
...
Clock fails somewhat similarly though it apparently didn't try to upgrade: 
Process: com.android.deskclock
Flags: 0xc8be45
Package: com.android.deskclock v203 (2.0.3)
Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys
android.database.sqlite.SQLiteException: no such column: incvol (code 1): , while compiling: SELECT _id, hour, minutes, daysofweek, alarmtime, enabled, vibrate, message, alert, incvol, profile FROM alarms WHERE (enabled=1)
...
        at com.android.deskclock.AlarmProvider.query(AlarmProvider.java:73)
        at android.content.ContentProvider.query(ContentProvider.java:744)
        at android.content.ContentProvider$Transport.query(ContentProvider.java:199)
        at android.content.ContentResolver.query(ContentResolver.java:414)
        at android.content.ContentResolver.query(ContentResolver.java:357)
        at com.android.deskclock.Alarms.getFilteredAlarmsCursor(Alarms.java:165)
        at com.android.deskclock.Alarms.calculateNextAlert(Alarms.java:344)
        at com.android.deskclock.Alarms.setNextAlert(Alarms.java:417)
        at com.android.deskclock.Alarms.saveSnoozeAlert(Alarms.java:510)
        at com.android.deskclock.AlarmInitReceiver$1.run(AlarmInitReceiver.java:49)
...
The media player fails with a NullPointerException: 
Process: com.andrew.apollo:main
Flags: 0x98be65
Package: com.andrew.apollo v2 (1.1)
Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys
java.lang.NullPointerException
        at com.andrew.apollo.MusicPlaybackService.stop(MusicPlaybackService.java:878)
        at com.andrew.apollo.MusicPlaybackService.openCurrentAndMaybeNext(MusicPlaybackService.java:1048)
        at com.andrew.apollo.MusicPlaybackService.openCurrentAndNext(MusicPlaybackService.java:1031)
        at com.andrew.apollo.MusicPlaybackService.access$1500(MusicPlaybackService.java:74)
        at com.andrew.apollo.MusicPlaybackService$MusicPlayerHandler.handleMessage(MusicPlaybackService.java:2337)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:137)
        at android.os.HandlerThread.run(HandlerThread.java:61)
Fixing the problem Maybe I could have worked out how to upgrade the relevant databases myself, but I went for simpler solutions. The clock settings are easy to re-enter and most of the media player state is regenerated by scanning files on the SD card. So I just deleted those on the Relay: 
  rm -rf /data/data/com.android.deskclock
  rm -rf /data/data/com.android.providers.media
The contacts were what I really cared about, and there are actually specific menu items in Contacts to export and import those (using VCF format), side-stepping the database upgrade. So I did:
  1. Insert SD card in Blade
  2. Open Contacts and export to VCF (I forget where this is in the menus but it was easy to find)
  3. Remove broken Contacts database on Relay:
    rm -rf /data/data/com.android.providers.contacts
  4. Insert SD card in Relay
  5. Move exported contacts to internal storage:
    mv /storage/sdcard1/*.vcf /storage/emulated/legacy
  6. Open People and tap the menu key, 'Import/export', 'Import from storage', then the filename
This may not include all data, and in particular it doesn't seem to include attached photos. But that was good enough for me.

28 August 2013

Vincent Sanders: Men admire the man who can organize their wishes and thoughts in stone and wood and steel and brass.


I would probably not yet worthy of the admiration Emerson was alluding to but I do like to make things. As anyone who has read previous posts knows I have pretty much embraced the "do things, tell people" idea.

One small wrinkle is doing things needs somewhere to work. Since moving myself and the family to rented accommodation in Cambridge (swampy 3,500 year old English city, not the one in Massachusetts) I have been lacking space to do practical projects.

The main space
To fix this lack I have joined the cambridge makespace which, in addition to somewhere I can work gives me access to some tools I was previously unable to afford. The space gives practical training on the more complex machines (any tool can be dangerous if you do not use it correctly) which recently allowed my induction on the CNC router.

My instructor , Mark Mellors, who was good enough to give up some of his valuable making time to train me (and accidentally get his car stuck in a car park overnight by staying late) suggested that it was a good idea to have a design to try.

I decided to use this opportunity to actually create something useful (though now I re-read Marks message it did suggest a simple design...oopsy). I had been working at the electronics bench in previous weeks and been uncomfortable using the existing stools and chairs as they were either a bit high or unable to be adjusted high enough for the 80cm tall benches. I decided to design a 60cm tall stool for use at this workbench.

My initial idea was for a simple three leg stool, round top, three legs, how hard could it be? Initial research showed that showed that above 30cm the legs needed to be braced to each other. This is because the leg to seat joints simply cannot handle the stress caused by leverage which longer legs introduce.

I looked at the structure of several stools online and was initially drawn towards creating something like the IKEA Dalfred bar stool. It was discovered that the design would be easier to realise if it were made from sheet material which give a smaller challenge to a naive operator of a CNC router. Because of this simpler designs were researched and I eventually found a simple design I liked.

The design could not however be used directly as it was for imperial sized material which is not available in europe. I selected the QCAD open source CAD package and recaptured the design adjusting for the available 12mm plywood sheet material. This resulted in an imperial measurement design for metric materials.

Mark helped me transfer the DXF into the CAM software (vcarve pro 7, after I discovered the demo version of this software generates files which cannot be imported into the full edition!) and generate toolpaths for our machine. Once the toolpaths are saved to a USB stick (no modern conveniences like direct upload here) the job can be run on the machine.

Here I ran into reality, turns out that tolerances in imperial combined with lack of understanding how plywood reacts resulted in excessive play in the joints. This resulted in an unusable stool, which simply tried to rotate around its central axis and become flatpack. I had successfully turned 10 worth of plywood into some sawdust and a selection of useless shapes. On the other hand I did become competent with the router workflow so it was not a complete failure.

I was determined to make the design work so I decided to start from scratch with a similar design but entirely in metric. I performed some material research both online and practically (why yes i did spend an informative hour in several cambridge DIY shops with digital calipers, why do you ask?) it turns out that generally available 12mm thick plywood actually ranges between 11.9 and 12mm thick.

I did some test slot cuts of varying width and determined that the available stock can be "persuaded" to fit into a 11.8mm wide slot. This Interference fit joint is strong and removes the need for adhesive in most cases.

Second cut still attached to the bed. Leftovers of first attempt in the background
The sheet plywood material is readily available with a width of 1220mm and a height of 606mm (a full sheet is 2440mm long which is cut into four with a 4mm wide saw blade) so making the legs fit within a sheet and be close to the 60cm target should be possible.

I selected a suitable seat radius (175mm) and from that determined the minimal gap to the base ring with a 6mm end mill tool (20mm for two toolpaths and some separation) and hence the minimum suitable width of the base ring (50mm) giving a total radius of 245mm.

For the legs allowance was made for two joints of 30mm with 30mm separation between them the legs come out at 90mm width. If a 6mm space and a 6mm toolpath top and bottom of the sheet is accounted for a 582mm height (606 - 24) is available in which to fit the legs. The top of the leg which fits into the seat is 12mm tall leaving 570mm total leg height.

12mm plywood stool design
At this point I selected some arbitrary values for the leg positioning and angle (30mm from seat centre and 15 ) using a right angle triangle triganometry this produced a stool with a base of around 550mm or 100mm outside the radius of the seat. This seemed a pleasing shape and when the values for the holding ring were calculated it is at 104mm height which also seems to work out well.

This design was cut on the machine, lightly finished with some sandpaper and assembled. The legs slotted into the ring first and then the legs eased into the seat slots, the whole thing flipped and the seat hammered home onto the legs.

Anne Harrison volunteering to try the wobbly stool of doom
Success! It physically fit together and if you were brave enough you could sit on it. Unfortunately the plywood seemed to flex around the central axis of rotation in a rather alarming way, fine if you are under 70 kilos but not giving the impression of security most people want from their seating.

Ok, lets try with five legs instead of three (at least we can reuse the existing three legs)...nope still not good enough and another 10 gone.

18mm Plywood stool design
The others in the space suggested a few ideas to improve matters and the one I selected was to use 18mm plywood instead of 12mm, this should improve rigidity. There was a brief pause in proceedings to discover 18mm sheet is actually 17.7mm and needs a 17.5mm slot to make the interference fit work.

Completed 18mm stool
A swift redesign later altering the seat radius, gap, ring width and leg height to accommodate the new material and we have version 4 and it works without caveat. Tested up to 150Kg load without trouble, there is a small amount of flex still but nothing that feels worrying.

I finished the stool by rounding the seat top edge with a ball bearing rounding bit in a manual router and applying a couple of coats of gloss acrylic varnish. Finished stool is now doing service at the space.

In conclusion the final design allows someone with a CNC router to create a useful 580mm high fixed stool for 7.50 in timber plus cutter wear and varnish so maybe 8.50 total.

You can actually get five legs and a seat/ring out of a 1220x606 sheet and with intelligent arrangement a single 1220x2440 sheet will probably yield five or possibly six stools in total

I am making the design files of the proven 18mm version available (heck they are all there...but you have been warned, none of the other solutions produce a satisfactory result) under the MIT licence so anyone can reproduce. More pretty pictures are also available.

22 May 2013

Lisandro Damián Nicanor Pérez Meyer: Debian/Ubuntu packages caching and mobile workstations

Not so long ago I read Dmitrijs' blog post on how to configure apt-cacher-ng to advertise it's service using avahi. As I normally use my laptop in my home and at work, and both networks have apt-cacher-ng running, I decided to give it a try.

I have been administering apt-cacher-ng for three networks so far, and I really find it a useful tool. Then, thanks to the aforementioned blog post, I discovered squid-deb-proxy. I don't use squid, so it's not for my normal use case, but some people will surely find it interesting.

But I found it's client package to be really interesting. It will discover any service providing _apt_proxy._tcp through avahi and let apt use it. But then the package wasn't available in Debian. So, I contacted Michael Vogt to see if he was interested in putting at least the client in Debian's archive. He took the opportunity to upload the full squid-deb-proxy, so thanks a lot Michael :-)

I then filled a wishlist bug against apt-cacher-ng to provide the avahi configuration for publishing the service, which Eduard included in the last version of it. So thanks a lot Eduard too!

tl;dr
You know only need apt-cacher-ng >= 0.7.13-1 and avahi-daemon installed on your server and your mobile users just need squid-deb-proxy-client. Then the proxy autoconfiguration for apt will just work.

One again, thanks a lot to the respective maintainers for allowing this into Jessie :-)

Gotchas
Yes, there are still some rough edges. On one of the networks I'm behind a proxy. While configuring my machine to use apt-cacher-ng's service as a proxy trough apt.conf, apt-listbugs would just work. But now, using the service as discovered by squid-deb-proxy-client, apt-listbugs just times out. Maybe I need to fill some other bug yet...

26 April 2013

Vincent Sanders: When you make something, cleaning it out of structural debris is one of the most vital things you do.

Collabora recently had a problem with a project's ARM build farm. In a nice change of pace it was not that the kernel was crashing, nor indeed any of the software or hardware.
The ProblemInstead our problem was our build farm could best be described as "a pile of stuff" and we wanted to add more systems to it and have switched power control for automated testing.

Which is kinda where the Christopher Alexander quote comes into this. I suggested that I might be able to come up with a better, or at least cleaner, solution.
The IdeaA subrack with sub modulesPrevious experience had exposed me to the idea of using 19 inch subracks for mounting circuits inside submodules.

I originally envisaged the dev boards individually mounted inside these boxes. However preliminary investigation revealed that the enclosures were both expensive and used a lot of space which would greatly increase the rack space required to house these systems.

imx53 QSB eurocard carrier
I decided to instead look at eurocard type subracks with carriers for the systems. Using my 3D printer I came up with a carrier design for the imx53 QSB and printed it. I used the basic eurocard size of 100mm x 160mm which would allow the cards to be used within a 3U subrack.

Once assembled it became apparent that each carrier would be able to share resources like power supply, ethernet port and serial console via USB just as the existing setup did and that these would need to be housed within the subrack.
The Prototype
The carrier prototype was enough to get enough interest to allow me to move on to the next phase of the project. I purchased a Schroff 24563-194 subrack kit and three packs of guide rails from Farnell and assembled it.

Initially I had envisaged acquiring additional horizontal rails from Schroff which would enable constructing an area suitable for mounting the shared components behind the card area.

Rear profile for Schroff subrackUnfortunately Schroff have no suitable horizontal profiles in their catalog and are another of those companies who seem to not want to actually sell products to end users but rather deal with wholesalers who do not have their entire product range!

Printed rear profile for Schroff subrack
Undaunted by this I created my own horizontal rail profile and 3D printed some lengths. The profile is designed to allow a 3mm thick rear cover sheet attached with M2.5 mounting bolts and fit rack sides in the same way the other profiles do.

At this point I should introduce some information on how these subracks are dimensioned. A standard 19 inch rack (as defined in IEC 60297) has a width of 17.75 inches(450.85mm) between the vertical posts. The height is measured in U (1.75 inches)

A subrack must obviously fit in the horizontal gap while providing as much internal space as possible. A subrack is generally either 3 or 6 U high. The width within a subrack is defined in units called HP (Horizontal Pitch) which are 0.2 inches(5.08 mm) and subracks like the Schroff generally list 84 usable HP.

However we must be careful (or actually just learn from me stuffing this up ;-) as the usable HP is not the same thing as the actual length of the horizontal rails! The enclosures actually leave and additional 0.1 inch at either end giving a total internal width of 85HP (17 inches, 431.8 mm) which leaves 0.75 inches for the subrack sides and some clearance.

The Schroff subrack allows eurocards to be slotted into rails where the card centre line is on HP boundaries, hence we describe the width of a card in the slot in terms of HP

I cannot manufacture aluminium extrusions (I know it is a personal failing) nor produce more than 100 mm long length of the plastic profile on my printer.

Even if full lengths are purchased from a commercial service (120 euros for a pair including tax and shipping) the plastic does not have sufficient mechanical strength.

The solution I came up with was somewhat innovative, as an alternative a M5 bolt into a thread in the aluminium extrusion I used a 444mm long length of 4mm threaded rod with nuts at either end. This arrangement puts the extrusion under compression and gives it a great deal of additional mechanical strength as the steel threaded rod is very strong.

Additionally to avoid having to print enough extrusion for the entire length I used some 6mm aluminium tube as a spacer between 6HP(30.48mm) wide sections of the printed extrusion.

It was intended to use a standard modular PC power supply which is 150mm wide which is pretty close to 30HP (6 inches) so it was decided to have a 6HP section of rail at that point to allow a rear mounting plate for the PSU to be attached.

This gives 6HP of profile, 21HP(106.68mm) of tube spacer, 6HP of profile, 46HP(233.68 mm) of tube spacer and a final 6HP profile summing to our total of 85HP. Of course this would be unnecessary if a full continuous 85HP rail had been purchased, but 6 of 6 HP long profile is only 51 euro a saving of 70 euro.

To provide a flat area on which to mount the power switching, Ethernet switch and USB hubs I ordered a 170 x 431 mm sheet of 3mm thick aluminium from inspiredsteel who, while being an ebay company, were fast, cheap and the cutting was accurate.

Do be sure to mention you would prefer it if any error made the sheet smaller rather than larger or it might not fit, for me though they were accurate to the tenth of a mm! If you would prefer the rear section of the rack to be enclosed when you are finished, buy a second sheet for the top. For my prototype I only purchased a 170 x 280mm sheet as I was unsure if I wanted a surface under the PSU (you do, buy the longer sheet)

PC power supply mounted to back plateMounting the PSU was a simple case of constructing a 3 mm thick plate with the correct cutouts and mounting holes for an ATX supply. Although the images show the PSU mounted on the left hand side of the rack this was later reversed to improve cable management.

The subrack needed to provide Ethernet switch ports to all the systems. A TP-Link TL-SF1016DS 16-Port 10/100Mbps Switch was acquired and the switch board removed from its enclosure. The switch selected has an easily removed board and is powered by a single 3.3V input which is readily available from the ATX PSU.

Attention now returned to the eurocard carriers for the systems, the boards to be housed were iMX53 QSB and iMX6 SABRE Lite and a Raspberry Pi control system to act as USB serial console etc.

The carriers for both main boards needed to be 8HP wide, comprised of:
Although only 38 mm this is 7.5HP and fractions of an HP are not possible with the selected subrack.

With 8HP wide modules this would allow for ten slots, within the 84 usable HP, and an eleventh 4HP wide in which the Raspberry Pi system fits.

iMX6 SABRE Lite eurocard carrierCarrier designs for both the i.MX53 QSB and the i.MX6 SABRE Lite boards were created and fabricated at a professional 3D print shop which gave a high quality finish product and removed the perceived risk of relying on a personal 3D printer for a quantity of parts.

This resulted in changes in the design to remove as much material as possible as commercial 3D services charge by the cubic cm. This Design For Manufacture (DFM) step removed almost 50% from the price of the initial design.

i.MX53 QSB carriers with wiring loom
The i.MX6 design underwent a second iteration to allow for the heatsink to be mounted and not mechanically interfere with the hard drive (although the prototype carrier has been used successfully for a system that does not require a hard drive). The lesson learned here is to be aware that an design iteration or two is likely and that it is not without cost.

The initial installation was to have six i.MX53 and two i.MX6 this later changed to a five/four split, however the carrier solution allows for almost any combination, the only caveat (discovered later) is the imx53 carriers should be to the right hand side with the small 4HP gap at that end as they have a JTAG connector underneath the board which otherwise foul the hard drive of the next carrier.

Racked cards showing unwanted cable tails
A wiring loom was constructed for each board giving them a connector tail long enough to allow them to be removed. This was the wrong approach! if you implement this design (or when I do it again) the connector tails on the wiring loom should present all the connections to the rear at the same depth as the Ethernet connection.

The rack cables themselves should be long enough to allow the slides to be removed but importantly it is not desirable to have the trailing cable on the cards. I guess the original eurocard designers figured this out as they designed the cards around the standard fixed DIN connectors at the back of the card slots.

USB relay board with wiring loom attached
We will now briefly examine a misjudgement that caused the initially deployed solution to be reimplemented. As the design was going to use USB serial converters to access the serial console a USB connected relay board was selected to switch the power to each slot. I had previously used serial controlled relay boards with a USB serial convertor however these were no longer available.

Initial deployment with USB controlled relay board
All the available USB relay boards were HID controlled, this did not initially seem to be an issue and Linux software was written to provide a reasonable interface. However it soon became apparent that the firmware on the purchased board was very buggy and crashed the host computer's USB stack multiple times.

Deployed solutionOnce it became apparent that the USB controlled power board was not viable a new design was conceived. As the Ethernet switch had ports available Ethernet controlled relay boards were acquired.

Evolution of 3mm PCB pillars
It did prove necessary to design and print some PCB support posts with M3 nut traps to allow the relay boards to be easily mounted using double sided adhesive pads.

By stacking the relay boards face to face and the Ethernet switch on top separated using nylon spacers it was possible to reduce the cable clutter and provide adequate cable routing space.

A busbar for Ground (black) and unswitched 12V (yellow) was constructed from two lengths of 5A chock block.

An issue with power supply stability was noted so a load resistor was added to the 12V supply and an adhesive thermal pad used to attach it to the aluminium base plate.

Completed redesign
It was most fortunate that the ethernet switch mounting holes lined up very well with the relay board mounting holes allowing for a neat stack.

This second edition is the one currently in use, it has proved reliable in operation and has been successfully updated with additional carriers.

The outstanding issues are mainly centered around the Raspberry Pi control board:
  • Needs its carrier fitting. It is currently just stuck to the subrack end plate.
  • Needs its Ethernet cable replacing. The existing one has developed a fault post installation.
  • Needs the USB hub supply separating from the device cable. The current arrangement lets the hub power the Pi which means you cannot power cycle it.
  • Connect its switched supply separately to the USB hub/devices.
Shopping listThe final bill of materials (excluding labour and workshop costs) which might be useful to anyone hoping to build their own version.

Prices are in GBP currency converted where appropriate and include tax at 20% and delivery to Cambridge UK and were correct as of April 2013.

The purchasing was not optimised and for example around 20GBP could be saved just by ordering all the shapeways parts in one order.
Base subrack
ItemSupplierQuantityLine Price
Schroff 24563-194 subrack kitFarnell141.28
Schroff 24560-351 guide railsFarnell313.65
Schroff rack rear horizontal railShapeways2100.00
1000mm length of 4mm threaded rodB and Q11.48
170mm x 431mm x 3mm Aluminium sheetinspired steel240.00
PSU mounting plateShapeways135.42
PCB standoffShapeways422.30
160mm Deep Modular PC supplyCCL155.76
TP-Link TL-SF1016DS 16-Port 10/100Mbps-SwitchCCL123.77
8 Channel 16A Relay Board Controlled Via EthernetRapid2126.00
Raspberry PiFarnell126.48
USB Serial convertersCCL1037.40
10 port strip style USB HUBEbay17.00
Parts for custom Ethernet cablesRS1326.00
Parts for custom molex power cables (salvaged from scrap ATX PSU)Workshop1111.00
33R 10W wirewound resistor for dummy loadRS11.26
24pin ATX female connector pre-wiredMaplin12.99
Akasa double sided thermal padMaplin15.00
Small cable tie basesMaplin16.49
Miscellaneous cable, connectors, nylon standoffs, solder, heatshrink, zip ties, nuts, washers etc. Workshop120.00
Total for subrack603.28

The carriers are similarly not optimally priced as over five GBP each can be saved by combining shipping on orders alone. Also the SSD drive selection was made some time ago and a newer model may be more suitable.
i.MX53 QSB carrier
ItemSupplierQuantityLine Price
i.MX53 QSBFarnell1105.52
Intel 320 SSD 80GCCL1111.83
Carrier boardShapeways130.00
combined sata data and power (15 to 20cm version)EBay15.00
Low profile right angle 5.5mm x 2.1mm barrel jackEBay10.25
Parts for 9pin serial cable extensionRS15.00
Miscellaneous solder, heatshrink, nylon nuts, bolts and washersWorkshop15.00
Total for carrier262.60

i.MX6 SABRE Lite carrier
ItemSupplierQuantityLine Price
i.MX6 SABRE LiteFarnell1128.06
Intel 320 SSD 80GCCL1111.83
Carrier boardShapeways135.00
combined sata data and power (15 to 20cm version)EBay15.00
Low profile right angle 5.5mm x 2.1mm barrel jackEBay10.25
Parts for 9pin serial cable modificationRS12.00
Miscellaneous solder, heatshrink, nylon nuts, bolts and washersWorkshop15.00
Total for carrier287.14
ConclusionThe solution works and in a 3U high 355mm deep subrack ten ARM development boards can be racked complete with local ethernet switching, power control and serial consoles.

Deployed system in situ configured as a build and test farm
The solution is neat and provides flexibility, density and reproducibility the "pile of stuff" solution failed to do.

For current prototype with nine filled slots the total cost was around 3000GBP or around 330GBP per slot which indicates a 100GBP per slot overhead over the "pile of stuff" solution. These figures omit the costs of the engineer and workshop time, which are estimated at an additional 1500GBP. Therefore a completed rack, fully filled with i.MX6 carriers costs around 5000GBP

Density could be increased if boards with lower height requirements were used however above twelve units there issues with Ethernet switch, power switch and USB port availability become a factor. For Example the 16 port Ethernet switch requires a port for uplink, one for each relay board and one for the console server which leaves only 12 ports for systems.

Addressing the outstanding issues would result in a much more user friendly solution. As the existing unit is in full time use and downtime is not easily scheduled for all ten systems, the issues are not likely to be fixed on the prototype and would have to be solved on a new build.

The solution is probably not suitable for turning into a product but that was not really the original aim. A commercial ARM blade server using this format would almost certainly use standard DIN connectors and a custom PCB design rather than adapting existing boards.

4 April 2013

Bits from Debian: Improvements in Debian's core infrastructure

Thanks to a generous donation by Bytemark Hosting, Debian started deploying machines for its core infrastructure services in a new data center in York, UK. This hardware and hosting donation will allow the Debian Systems Administration (DSA) team to distribute Debian's core services across a greater number of geographically diverse locations, and improve, in particular, the fault-tolerance and availability of end-user facing services. Additionally, the storage component of this donation will dramatically reduce the storage challenges that Debian currently faces. The hardware provided by Bytemark Hosting consists of a fully-populated HP C7000 BladeSystem chassis containing 16 server blades: and several HP Modular Storage Arrays: with 108 drive bays in total, mostly 500GB SATA drives, some 2TB, some 600GB 15kRPM SAS, providing a total of 57 TB. 57 TB today could host roughly 80 times the current Debian archive or 3 times the Debian Snapshot archive. But remember both archives are constantly growing!

14 January 2013

Petter Reinholdtsen: Modalias strings - a practical way to map "stuff" to hardware

While looking into how to look up Debian packages based on hardware information, to find the packages that support a given piece of hardware, I refreshed my memory regarding modalias values, and decided to document the details. Here are my findings so far, also available in the Debian Edu subversion repository: Modalias decoded This document try to explain what the different types of modalias values stands for. It is in part based on information from <URL: https://wiki.archlinux.org/index.php/Modalias >, <URL: http://unix.stackexchange.com/questions/26132/how-to-assign-usb-driver-to-device >, <URL: http://code.metager.de/source/history/linux/stable/scripts/mod/file2alias.c > and <URL: http://cvs.savannah.gnu.org/viewvc/dmidecode/dmidecode.c?root=dmidecode&view=markup >. The modalias entries for a given Linux machine can be found using this shell script: 
find /sys -name modalias -print0   xargs -0 cat   sort -u
The supported modalias globs for a given kernel module can be found using modinfo: 
% /sbin/modinfo psmouse   grep alias:
alias:          serio:ty05pr*id*ex*
alias:          serio:ty01pr*id*ex*
%
PCI subtype A typical PCI entry can look like this. This is an Intel Host Bridge memory controller:
pci:v00008086d00002770sv00001028sd000001ADbc06sc00i00
This represent these values: 
 v   00008086  (vendor)
 d   00002770  (device)
 sv  00001028  (subvendor)
 sd  000001AD  (subdevice)
 bc  06        (bus class)
 sc  00        (bus subclass)
 i   00        (interface)
The vendor/device values are the same values outputted from 'lspci -n' as 8086:2770. The bus class/subclass is also shown by lspci as 0600. The 0600 class is a host bridge. Other useful bus values are 0300 (VGA compatible card) and 0200 (Ethernet controller). Not sure how to figure out the interface value, nor what it means. USB subtype Some typical USB entries can look like this. This is an internal USB hub in a laptop:
usb:v1D6Bp0001d0206dc09dsc00dp00ic09isc00ip00
Here is the values included in this alias: 
 v    1D6B  (device vendor)
 p    0001  (device product)
 d    0206  (bcddevice)
 dc     09  (device class)
 dsc    00  (device subclass)
 dp     00  (device protocol)
 ic     09  (interface class)
 isc    00  (interface subclass)
 ip     00  (interface protocol)
The 0900 device class/subclass means hub. Some times the relevant class is in the interface class section. For a simple USB web camera, these alias entries show up:
usb:v0AC8p3420d5000dcEFdsc02dp01ic01isc01ip00
usb:v0AC8p3420d5000dcEFdsc02dp01ic01isc02ip00
usb:v0AC8p3420d5000dcEFdsc02dp01ic0Eisc01ip00
usb:v0AC8p3420d5000dcEFdsc02dp01ic0Eisc02ip00
Interface class 0E01 is video control, 0E02 is video streaming (aka camera), 0101 is audio control device and 0102 is audio streaming (aka microphone). Thus this is a camera with microphone included. ACPI subtype The ACPI type is used for several non-PCI/USB stuff. This is an IR receiver in a Thinkpad X40:
acpi:IBM0071:PNP0511:
The values between the colons are IDs. DMI subtype The DMI table contain lots of information about the computer case and model. This is an entry for a IBM Thinkpad X40, fetched from /sys/devices/virtual/dmi/id/modalias:
dmi:bvnIBM:bvr1UETB6WW(1.66):bd06/15/2005:svnIBM:pn2371H4G:pvrThinkPadX40:rvnIBM:rn2371H4G:rvrNotAvailable:cvnIBM:ct10:cvrNotAvailable:
The values present are 
 bvn  IBM            (BIOS vendor)
 bvr  1UETB6WW(1.66) (BIOS version)
 bd   06/15/2005     (BIOS date)
 svn  IBM            (system vendor)
 pn   2371H4G        (product name)
 pvr  ThinkPadX40    (product version)
 rvn  IBM            (board vendor)
 rn   2371H4G        (board name)
 rvr  NotAvailable   (board version)
 cvn  IBM            (chassis vendor)
 ct   10             (chassis type)
 cvr  NotAvailable   (chassis version)
The chassis type 10 is Notebook. Other interesting values can be found in the dmidecode source: 
  3 Desktop
  4 Low Profile Desktop
  5 Pizza Box
  6 Mini Tower
  7 Tower
  8 Portable
  9 Laptop
 10 Notebook
 11 Hand Held
 12 Docking Station
 13 All In One
 14 Sub Notebook
 15 Space-saving
 16 Lunch Box
 17 Main Server Chassis
 18 Expansion Chassis
 19 Sub Chassis
 20 Bus Expansion Chassis
 21 Peripheral Chassis
 22 RAID Chassis
 23 Rack Mount Chassis
 24 Sealed-case PC
 25 Multi-system
 26 CompactPCI
 27 AdvancedTCA
 28 Blade
 29 Blade Enclosing
The chassis type values are not always accurately set in the DMI table. For example my home server is a tower, but the DMI modalias claim it is a desktop. SerIO subtype This type is used for PS/2 mouse plugs. One example is from my test machine:
serio:ty01pr00id00ex00
The values present are 
  ty  01  (type)
  pr  00  (prototype)
  id  00  (id)
  ex  00  (extra)
This type is supported by the psmouse driver. I am not sure what the valid values are. Other subtypes There are heaps of other modalias subtypes according to file2alias.c. There is the rest of the list from that source: amba, ap, bcma, ccw, css, eisa, hid, i2c, ieee1394, input, ipack, isapnp, mdio, of, parisc, pcmcia, platform, scsi, sdio, spi, ssb, vio, virtio, vmbus, x86cpu and zorro. I did not spend time documenting all of these, as they do not seem relevant for my intended use with mapping hardware to packages when new stuff is inserted during run time. Looking up kernel modules using modalias values To check which kernel modules provide support for a given modalias, one can use the following shell script: 
  for id in $(find /sys -name modalias -print0   xargs -0 cat   sort -u); do \
    echo "$id" ; \
    /sbin/modprobe --show-depends "$id" sed 's/^/  /' ; \
  done
The output can look like this (only the first few entries as the list is very long on my test machine): 
  acpi:ACPI0003:
    insmod /lib/modules/2.6.32-5-686/kernel/drivers/acpi/ac.ko 
  acpi:device:
  FATAL: Module acpi:device: not found.
  acpi:IBM0068:
    insmod /lib/modules/2.6.32-5-686/kernel/drivers/char/nvram.ko 
    insmod /lib/modules/2.6.32-5-686/kernel/drivers/leds/led-class.ko 
    insmod /lib/modules/2.6.32-5-686/kernel/net/rfkill/rfkill.ko 
    insmod /lib/modules/2.6.32-5-686/kernel/drivers/platform/x86/thinkpad_acpi.ko 
  acpi:IBM0071:PNP0511:
    insmod /lib/modules/2.6.32-5-686/kernel/lib/crc-ccitt.ko 
    insmod /lib/modules/2.6.32-5-686/kernel/net/irda/irda.ko 
    insmod /lib/modules/2.6.32-5-686/kernel/drivers/net/irda/nsc-ircc.ko 
  [...]
If you want to help implementing a system to let us propose what packages to install when new hardware is plugged into a Debian machine, please send me an email or talk to me on #debian-devel. Update 2013-01-15: Rewrite "cat $(find ...)" to "find ... -print0 xargs -0 cat" to make sure it handle directories in /sys/ with space in them.

24 July 2012

Gunnar Wolf: Ten theses in favor of free download of cultural goods on the Internet (by Enrique G. Gallegos)

This is one of the days where reading my everyday newspaper was worth more than just getting bitter at the news. I found this text in La Jornada, my usual newspaper. I liked it very much, and decided to translate it for a wider audience. Of course, if you can read Spanish, do yourself a favor and go to the original. It is not that the text is so easy to translate. And, after all, I'm not a native English writer. I'm trying to do a literal translation, even when disagreeing with the author. Ten theses in favor of free download of cultural goods on the Internet Enrique G. Gallegos Poet and philosopher. Currently a researcher in Universidad Aut noma Metropolitana-C
  1. Historical legacy. Criticists who promote the persecution of piracy and free downloads of cultural works from the Internet argue that the patrimonial rights have been wronged; they start off suposing that a cultural work is formed from a historical void, as if there was a "nothing" to begin with, and then a "something" appeared. Nothing more fallacious: All cultural products has its precedents, and thanks to these it generates part of its best forces.
  2. Opening towards the future. Human beings are projected into the future. As the main philosophies of the 20th century have explored, one of the singular characteristics of mankind is the ability to think and imagine the future. Products of culture are the best ways to think and project society, politics, love, hope, needs, failures A song, a poem, an essay, can trigger imaginary worlds with a transformative potential. To deny this to humanity with "patrimonial damage" as an argument is to mutilate man's temporal nature.
  3. Recognition of the present. Cultural works generate moments of reflection, criticism and pleasure; they can trigger actions demanding commitment, solidarity and strong indignation against injustice. If every cultural work is adscribed in some way to a historic tradition, and if cultural products anre necessary to imagine other possible worlds and open our future, it's not less true that they also allow us to recognize our present by making us more sensitive to others' ideas, sensations and emotions. Without culture, there is no present nor others' presence.
  4. Divulgation of the cultural heritage. If culture is "heritage of mankind", it should be also spread by any possible way. But not just spread, but actively sought for the biggest number of people to be able to effectively have access to it. In a potentially interconnected world, Internet is the most adequate world for it (despite its limitations). Therefore, penalizing free downloads means avoiding spreading it and restricting access to those groups who don't have enough economic resources.
  5. Preserving cultural heritage. If we agree that cultural works are heritage of mankind, we need to take every action needed to preserve them. But preserving culture does not only mean keeping the products in museums, galleries or never-polluted drawers; it rather means to keep them in the collective memory and in the flow of constant interpretations and appropiations. Said in other words, the genuine way to preserve culture is to allow universal access to cultural goods. That's why, holding that free copies and downloads of books, music, videos, etc. over the Internet are harmful, is an incompatible argument with the obligation to preserve cultural heritage.
  6. They are not goods. Cultural works are material and spiritual manifestations of mankind, concretions of its historicity; they are, also, expressions of emotions, ideas and projections of other lives and other worlds. That's why they cannot be equated to goods or inserted in patrimonialist logic. The market might want to cheat by giving a certain price to a painting by Orozco or auctioning a Baudelaire manuscript, but it will never be able to aprehend their true meaning as cultural works. And it's not that culture is not faluable, but taht its estimation criteria do not follow the rules of market, but those of the imponderable and unlimited.
  7. Price disproportion. Assuming we accept the possibility that intermediaries charge for the services they perform, the value of a book, record or movie should never exceed the daily wage of a worker or employee. But this ellection would only be one more option within the effective possibility that people opt to download or freely copy the cultural work. The final decision as to which media to go to should be a sovereign resolution of the culture-interested person.
  8. The principle of the most benefit. Even when copies and free downloads over Internet could generate a patrimonial "wrong" to third parties, the cultural benefit obtained by it will always be greater in as much that it carries out intensely the principle to foster, spread and access the cultural heritage of mankind. To think otherwise is to privilege the few over the many.
  9. The real evil is elsewhere. Murder, women- and children-trade, poverty and misery are the real evils that ache mankind. According to the United Nations, in 2010 there were 468 thousand murders in the world; the estimation is that 3500 million people live in poverty; in their 2009 inform, the UN found over 2400 victims of "people trade", kidnapped as sex slaves; in some regions in Africa, close to 30% of children suffer acute malnutrition and six people (between children and adults) die every day from hunger. That's why pretending that free download of cultural works is an evil is clearly a tricky and deceptive speech
  10. Against the intermediary-salesman. Culture does not need intermedieries that reduce cultural works to goods. Culture is too important to leavie it in the hands of salespeople that simplistly equate cultural goods with cakes. An intermediary of this kind will never understand the difference between a work of art and a disposable razor blade. What the world needs is more support from the government from every country to artists, creators and poets, as well as ensuring the conditions for the absolute mobility freedom for cultural works.

31 May 2012

Russell Coker: Links May 2012

Vijay Kumar gave an interesting TED talk about autonomous UAVs [1]. His research is based on helicopters with 4 sets of blades and his group has developed software to allow them to develop maps, fly in formation, and more. Hadiyah wrote an interesting post about networking at TED 2012 [2]. It seems that giving every delegate the opportunity to have their bio posted is a good conference feature that others could copy. Bruce Schneier wrote a good summary of the harm that post-911 airport security has caused [3]. Chris Neugebauer wrote an insightful post about the drinking culture in conferences, how it excludes people and distracts everyone from the educational purpose of the conference [4]. Matthew Wright wrote an informative article for Beyond Zero Emissions comparing current options for renewable power with the unproven plans for new nuclear and fossil fuel power plants [5]. The Free Universal Construction Kit is a set of design files to allow 3D printing of connectors between different types of construction kits (Lego, Fischer Technic, etc) [6]. Jay Bradner gave an interesting TED talk about the use of Open Source principles in cancer research [7]. He described his research into drugs which block cancer by converting certain types of cancer cell into normal cells and how he shared that research to allow the drugs to be developed for clinical use as fast as possible. Christopher Priest wrote an epic blog post roasting everyone currently associated with the Arthur C. Clarke awards, he took particular care to flame Charles Stross who celebrated The Prestige of such a great flaming by releasing a t-shirt [8]. For a while I ve been hoping that an author like Charles Stross would manage to make more money from t-shirt sales than from book sales, Charles is already publishing some of his work for free on the Internet and it would be good if he could publish it all for free. Erich Schubert wrote an interesting post about the utility and evolution of Favebook likes [9]. Richard Hartmann wrote an interesting summary of the problems with Google products that annoy him the most [10]. Sam Varghese wrote an insightful article about the political situation in China [11]. The part about the downside of allowing poorly educated people to vote seems to apply to the US as well. Sociological Images has an article about the increased rate of Autism diagnosis as social contagion [12]. People who get their children diagnosed encourage others with similar children to do the same. Vivek wrote a great little post about setting up WPA on Debian [13]. It was much easier than expected once I followed that post. Of course I probably could have read the documentation for ifupdown, but who reads docs when Google is available? Related posts:
  1. Links March 2012 Washington s Blog has an informative summary of recent articles about...
  2. Links April 2012 Karen Tse gave an interesting TED talk about how to...
  3. Links February 2012 Sociological Images has an interesting article about the attempts to...

7 April 2012

Bastian Blank: Booting Debian via AoE (ATA over Ethernet) and PXE

AoE is one of the protocols supported by Linux to access storage via network. It uses plain Ethernet for communication and includes a discovery mechanism to find all available targets. I use it to provide disk space to VMs running on different machines. Next step is to boot via AoE. It is no real problem to use AoE in running systems. However with some help it is even possible to actually boot disk-less machines via AoE. The PXE implementation iPXE provides the AoE support to actually boot from. I will describe the necessary parts.
Setup vblade The AoE target used it vblade. vblade needs access to raw sockets. As I prefer to not have run anything as root if it is not necessary, I use filesystem capabilities to allow it access to the network. 
setcap cap_net_raw+ep /usr/sbin/vblade
vblade gets the mac address of the initiator, the shelf and slot number, the network device and the block device. 
/usr/sbin/vblade -m $mac 0 0 eth0 $dev
Setup a tftp server and iPXE 
apt-get install atftpd ipxe
ln -s /usr/lib/ipxe/undionly.kpxe /var/lib/tftpboot
Setup ISC dhcpd The dhcp server needs to be configured. It needs to hand out two distinct parameter sets. The first is used to chain-load iPXE into the normal PXE stack. The second is for iPXE and sets the root path to the AoE device. They are selected on the iPXE marker in the request. 
if exists user-class and option user-class = "iPXE"  
  filename "";
  option root-path "aoe:e0.0";
  else  
  filename "undionly.kpxe";
Support AoE via initramfs-tools The initramfs needs to initialize AoE support. It needs to enable the network device used for communication with the AoE server and wait until it is up. After that it needs to load the aoe module and run aoe-discover. We should have all devices now. The root device can now be used like any other normal device. After the AoE device is initialized, it can be found via UUID and all the other ways. So no further modifications are necessary over the usage of local disks. The initramfs finds the device as usual and boots from it. The initramfs support is still a prototyp, but seems to work. For initramfs-tools it needs a hook to include all necessary stuff in the initramfs and a script to actually do the work. Both are shown here. /etc/initramfs-tools/hooks/aoe: 
#!/bin/sh
case $1 in
prereqs)
  echo "udev"
  exit 0
  ;;
esac
. /usr/share/initramfs-tools/hook-functions
copy_exec /sbin/aoe-discover
manual_add_modules aoe
/etc/initramfs-tools/scripts/local-top/aoe: 
#!/bin/sh
case $1 in
prereqs)
  echo "udev"
  exit 0
  ;;
esac
ifconfig eth0 up
sleep 10
modprobe aoe
aoe-discover
udevadm settle --timeout=30
Problems Not all parts of this works 100%. Some parts works not for all hardware.
  • My old notebook is not able to run Linux with this setup. grub loads the kernel via AoE and nothing comes later.
  • The network may need more time. Especially in large environments with spanning tree enabled, it may need half a minute until any packets will flow.
Some of the problems can be addressed later. Some can't.

5 April 2012

Steve Kemp: So I want a backup solution

I look after a lot of systems, and most of them want identical and simple backups taking of their filesystems. Currently I use backup2l which works but suffers from a couple of minor issues. In short I want to take a full filesystem backup (i.e. Backup "/"). I wish to only exclude a few directories and mounted filesystems. So my configuration looks like this: 
# List of directories to make backups of.
# All paths MUST be absolute and start with a '/'!
SRCLIST=(  / /boot -xdev )
# The following expression specifies the files not to be archived.
SKIPCOND=( -path '/var/backups/localhost' -o -path '/var/run/' -o \
    -path '/tmp'  -o -path '/var/tmp' \
    -o -path '/dev' -o -path '/spam' \
    -o -path '/var/spool/' )
The only surprising thing here is that I abuse the internals of backup2l because I know that it uses "find" to build up a list of files - so I sneakily add int "-xdev" to the first argument. This means I don't accidentally backup any mounted gluster filesystem, mounted MySQL binary/log mounts, etc. backup2l then goes and does its jobs. It allows me to define things to run before and after the backup runs via code like this: 
# This user-defined bash function is executed before a backup is made
PRE_BACKUP ()
 
   if [ -d /etc/backup2l/pre.d/ ]; then
      run-parts /etc/backup2l/pre.d/
   fi
So what is my gripe? Well I get a daily email, per-system, which shows lots of detail - but the key thing. The important thing. The thing I care about more than anything else, the actual "success" or "fail" result is only discoverable by reading the mail. If the backup fails, due to out of disk, I won't know unless I read the middle of the mail. If the pre/post-steps fail I won't know unless I examine the output. As I said to a colleague today in my view the success or failure of the backup is the combination of each of three distinct steps: If any of the three fail I want to know. If they succeed then ideally I don't want a mail at all - but if I get one it should have: 
Subject: Backup Success - $(hostname) - $(date)
So I've looked around at programs such as backup-ninja, backup-manager and they seem similar. It is a shame as I mostly like backup2l, but in short I want to do the same thing on about 50-250 hosts: In my case it is usually the rsync-step that fails. Which is horrific if you don't notice (quota exceeded. connection reset by peer. etc). The local backups are good enough for 95% of recovery times - but if the hardware is fried having the backups be available, albeit slowly, is required. Using GNU Tar incrementally is trivial. If it weren't such a messy program I'd probably be inclined to hack on backup2l - but in 2012 I can't believe I need to. (Yes, backuppc rocks. So does duplicity. So does amanda. But they're not appropriate here. Sadly.) ObQuote: "Oh, I get it. I see now. You've been training for two years to take me out, and now here I am. Whew! " - Blade II - An example of a rare breed, a sequel that doesn't suck. No pun intended.

25 November 2011

Marco d'Itri: How to reset a confused iLO

Due to some still unknown bug, all the iLOs of two of my HP blade enclosures became confused and stopped accepting TCP connections. This not only made them unusable, but also prevented upgrading the firmware to an hopefully fixed version... My solution: reset them from the OS by running: 
ipmitool mc reset cold
HP's support solution: remove and reseat each blade.

21 July 2011

Wouter Verhelst: On the beard, again...

It looks like I might actually lose it sooner than I was thinking in my previous blog post, a few days ago. On the 18th, Formateur Elio di Rupo had written a proposed document on which to start negotiations. Most parties agreed to that, except for N-VA and CD&V. Unfortunately, that was enough to make further negotiations useless, so di Rupo had offered his resignation as formateur to the King, who kept his response "under advisement". I thought I was going to end up like Andrew. Since he wasn't formally ex-formateur yet, however, he continued working. And today, merely a few days later, with an extra proposal and perhaps also partly due to the King's angry speech for today's national holiday CD&V has now been persuaded to join the negotiations. It's been over fourhundred days, and far too many weirdos, but they're actually sitting down to talk. Like, right now. I'll hold off on buying those razor blades for now, but we might yet get there.

29 January 2011

Adrian von Bidder: Sci-Fi classics (and other stuff)

I again find myself spending time watching movies ... catching up on all those friends who chastise me for not having seen whatever movie we're talking about. For example, the first three Terminator movies. I'm always fond of old style science fiction, so the first of these is quite cute. And then there's the liquid metal effect introduced in Judgment Day, which also is cool, and at the time certainly was at the bleeding edge of what was possible in CG. But by the third movie the concept itself is sure showing its age although it's still well made action, I didn't enjoy it as much. Haven't got hold of the Salvation nor Bruno Mattai's unofficial Terminator 2. Speaking about old fashioned Sci Fi: Blade Runner has a very nice retro look in the buildings and furnishings. And, although this is more because it also was made in the early 80s, very retro computer consoles :-) Fun to watch, but I'm not quite happy with how the plot turns out in the end. But then, it's a Hollywood production, so should I be surprised? But there's not only Sci Fi ... The Shawshank Redemption is a gripping story about an innocent serving a 20 years sentence for murder. A banker entering the rough world of prison, bets are taken how long he'll last. But obviously it turns out that he's really quite tough ... I'm getting good at this ... I thought Jim Jarmusch after about 20s of the opening credits of Down By Law. While it is (again) about innocent people in prison, the focus here is solely on the interaction of three people sharing a cell, excluding almost everything else. Done beautifully in black and white, and while it's not fast paced in any way, the plot has a steady flow to it.

Next.

Previous.